REMARKS 

Claims 1-37 are pending in the application and stand rejected. 
Drawing Objections 

In response to the drawing objections, Applicant submits herewith a set of formal 
drawings for Figs. 1-9. Withdrawal of the objection is thus requested. 
Claim Rejections - 35 U.S.C. § 102 

Claims 1-9 and 11-37 stand rejected under 35 U.S.C. § 102(e) as being anticipated by 
U.S. Patent No. 6,1 19, 147 to Toomev . Applicant respectfully traverses the rejection. 

Under 35 U.S.C. 102, a claim is anticipated only if each and every element as set forth in 
the claim is found, either expressly or inherently described, in a single prior art reference. The 
identical invention must be shown in as complete detail as is contained in the claim. (See MPEP 
§2131). The single prior art reference must disclose all of the elements of the claimed invention 
functioning essentially in the same manner (see, Shanklin Corp. v. Springfield Photo Mount 
Corp, 521 F.2d 609 (1 st Cir. 1975). 

Here, it is respectfully submitted that at the very minimum, Toomev does not disclose or 
suggest elements of the inventions of independent claims 1,19 and 29. Indeed, on a fundamental 
level , one of ordinary skill in the art would readily recognize that the teachings of Toomev are 
unrelated to the claimed inventions, and clearly do no anticipate the claimed inventions. 

In particular, Toomev is directed to a system for supporting a collaborative work 
environment, which enables a plurality of participants to review and augment meetings that take 
place in a virtual environment (see, Col. 1 , lines 8-14 and lines 47-54). More specifically, in 
Figs. 1 and 2, Toomev discloses a computer-mediated asynchronous meeting system (100) that 
enables a plurality of participants (who are remotely located in space and/or time from each 
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other) to asynchronously interact with a multi-modal document (105) through a plurality of 
Uls/displays (120) connected to the system (see, e.g., Col. 5, lines 1 1-23). 

Toomey further discloses that the multi-modal document (105) is a data structure that 
stores a combination of the participants' collaborative interactions, i.e., the multi-modal 
document (105) captures all the meeting events of participants to the meeting. More specifically, 
the multi-modal document (105) comprises a combination of data structures associated with a 
related virtual meeting, including text discussion, audio recordings, graphics, documents and 
agendas, to provide a complete context of a meeting for subsequent participants (see, Col. 5, 
lines 24-32; Col 6, lines 5-21). The Toomey system (Fig. 2) includes an augmentation 
controller (220) allowing annotation to the multi-modal document (105) associated with a given 
meeting. 

Toomey clearly does not anticipate the invention of claim 1 . Claim 1 is directed to a 
system comprising: 

a multi-modal application comprising at least a first mode process and a second mode 
process; 

a multi-modal shell for managing and synchronizing information exchanges between the 
first and second mode processes of the multi-modal application; and 

an API (application program interface) that allows the first and second mode processes 
to register their respective active commands and corresponding actions with the multi- modal 
shell. 

Examiner contends that the multiple user interfaces/displays (120) of Fig. 2 of Toomey 
disclose a multi-modal application with a first and second mode processes. Applicant 
respectfully disagrees with this characterization. A multi-modal application is an application 
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that can be controlled by commands in two or more modalities, e.g., an e-mail application that 
allows a user to access e-mails by a voice commands or GUI commands. The Ul/displays (120) 
of Toomey are nothing more than clients that enable users to, e.g., input meeting events (e.g., 
text inputs, prerecorded sound files, etc) that are recorded by the system in a multi-modal data 
structure (i.e., the MM document (105)). 

Further, although Toomey discloses that each user can run and instance of a "client 
software program to join others in virtual space" (see, Col. 6, lines 38-41), there is nothing in 
Toomey that discloses that such "client software program" is a multi-modal application that 
comprises a first mode process and a second mode process . Indeed, although Toomey 
discloses that a user can provide input events such as text utterances (i.e., text strings") or sound 
cues (i.e., prerecorded sound) via the "client software program" (see, Col. 6, lines 38-65) that are 
recorded in the multi-modal document (105), there is nothing in Toomey that remotely discloses 
or suggest that the "client software program" can be commanded by different modes. 

Examiner further contends that the multi-modal capture controller (230) of Toomey 
discloses a multi-modal shell for managing and synchronizing information exchanges between 
the first and second mode processes of the multi-modal application, as recited in claim 1 . 
Applicant respectfully disagrees with this characterization, Toomey expressly discloses that the 
multi-modal capture controller (230) is a module that captures and records the different input 
events (text inputs, sound files, etc.) that are provided by participants during a meeting 
(recording of each event in the multi-modal document) (see, e.g., Col. 5, lines 55-59; and Col. 
16, lines 60-61). There is nothing in Toomey that discloses or suggests that the multi-modal 
capture controller (230) manages and synchronizes information exchanges between a first and 
second mode processes of a multi-modal application. 
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In fact, even assuming, arguendo, the validity of Examiner's characterization of the 
Ul/displays (120) as being a "multi-modal application" with a first and second mode process, 
Examiner has not even demonstrated how the multi-modal capture controller (230) manages and 
synchronizes information exchanges between the different Uls/displays (120). In any event, it is 
respectfully submitted that Toomev does not even suggest that the multi-modal capture controller 
(230) manages and synchronizes information exchanges between the different Uls/displays 
(120). 

Examiner further contends that the I/O interface (210) of Toomev discloses an API 
(application program interface) that allows the first and second mode processes to register their 
respective active commands and corresponding actions with the multi- modal shell, as recited in 
claim 1 . Applicant respectfully disagrees with this characterization. Toomev expressly 
discloses that the I/O interface (21) simply routes inputs/outputs to/from the displays (120) and 
different controllers (220, 230 and 240), depending on whether the system is capturing a 
meeting, augmenting a previously captured meeting or replaying a meeting (see, Col. 5, lines 48- 
55). There is simply nothing in Toomev that discloses or suggests that the I/O interface (201) 
allows the first and second mode processes (i.e., the displays (120) as characterized by 
Examiner) to register their respective active commands and corresponding actions with the 
multi- modal shell (i.e., the capture controller (230) as characterized by Examiner). More 
specifically, by way of example, Toomev does not suggest that the I/O interface (21) registers 
active commands and corresponding actions - - the I/O interface merely routes I/O events of the 
system . 
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Moreover, with regard to claims 19 and 29, Toomey clearly does not disclose or suggest 
the methods steps of: 

receiving a command in a first modality; 

triggering (i) an action in the first modality and (ii) a corresponding action in at least a 
second modality, based on the received command; and 

updating application states or device states associated with the first modality and the 
second modality. 

In rejecting claims 19 and 29, Examiner relies essentially on the same basis for the 
rejection of claims 1 and 2. However, Examiner does not specifically address (or even point to 
portions of Toomey that disclose or suggest) the claimed steps of receiving a command in a first 
modality or triggering (i) an action in the first modality and (ii) a corresponding action in at 
least a second modality, based on the received command, as recited in claims 19 and 29. Thus, 
the rejection of claims 19 and 29 appears to be legally deficient on its face. 

Furthermore, it is respectfully submitted that Examiner's reliance on Col. 2, lines 56-57 
and Col. 9, line 55 of Toomey as disclosing the step of updating application states or device 
states associated with the first modality and the second modality, is misplaced. The "updating 
of a state" a disclosed by Toomey is nothing more than the process of updating a multi-modal 
document representation of a meeting by adding or replacing recorded meeting events, 
updating 

In stark contrast, the inventions of claims 19 and 29 are directed to methods for 
synchronizing multi-modal interactions wherein the application or device states of applications 
or devices of different modalities are updated in response to a command executed in a first 
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modality, which triggers the execution of a corresponding command in a second modality. 
Toomey is not even remotely related to the inventions of claims 19 and 29. 

Therefore, for at least the above reasons, claims 1,19 and 29 (and all pending rejected 
claims that depend from claims 1, 19 or 29) are patentably distinct and patentable over Toomey . 
Accordingly, withdrawal of the claim rejections under 35 U.S.C. 102 is respectfully requested. 
Claim Rejections - 35 U.S.C. § 103 

Claim 10 stands rejected under 35 U.S.C. § 103(a) as being anticipated by Toomey . 
Claim 10 depends from claim 1. Claim 10 is patentable and non-obvious over Toomey for at 
least the same reasons give above for claim 1. Therefore, withdrawal of the rejection is 
respectfully requested. 
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